Feedback-assisted automatic target and clutter discrimination using a Bayesian convolutional neural network for improved explainability in SAR applications

DATA AVAILABILITY STATEMENT : The NATO-SET 250 dataset is not publicly available; however, the MSTAR dataset can be found at the following url: https://www.sdms.afrl.af.mil/index.php?collection=mstar (accessed on 5 January 2022).In this paper, a feedback training approach for efficiently dealing with distribution shift in synthetic aperture radar target detection using a Bayesian convolutional neural network is proposed. After training the network on in-distribution data, it is tested on out-of-distribution data. Samples that are classified incorrectly with high certainty are fed back for a second round of training. This results in the reduction of false positives in the out-of-distribution dataset. False positive target detections challenge human attention, sensor resource management, and mission engagement. In these types of applications, a reduction in false positives thus often takes precedence over target detection and classification performance. The classifier is used to discriminate the targets from the clutter and to classify the target type in a single step as opposed to the traditional approach of having a sequential chain of functions for target detection and localisation before the machine learning algorithm. Another aspect of automated synthetic aperture radar detection and recognition problems addressed here is the fact that human users of the output of traditional classification systems are presented with decisions made by “black box” algorithms. Consequently, the decisions are not explainable, even to an expert in the sensor domain. This paper makes use of the concept of explainable artificial intelligence via uncertainty heat maps that are overlaid onto synthetic aperture radar imagery to furnish the user with additional information about classification decisions. These uncertainty heat maps facilitate trust in the machine learning algorithm and are derived from the uncertainty estimates of the classifications from the Bayesian convolutional neural network. These uncertainty overlays further enhance the users’ ability to interpret the reasons why certain decisions were made by the algorithm. Further, it is demonstrated that feeding back the high-certainty, incorrectly classified out-of-distribution data results in an average improvement in detection performance and a reduction in uncertainty for all synthetic aperture radar images processed. Compared to the baseline method, an improvement in recall of 11.8%, and a reduction in the false positive rate of 7.08% were demonstrated using the Feedback-assisted Bayesian Convolutional Neural Network or FaBCNN.The Radar and Electronic Warfare department at the CSIR.http://www.mdpi.com/journal/remotesensinghj2023Electrical, Electronic and Computer Engineerin

Improved explainability through uncertainty estimation in automatic target recognition of SAR images

In recent years, there has been significant developments in artificial intelligence (AI), with machine learning (ML) implementations achieving impressive performance in numerous fields. The defence capability of countries can greatly benefit from the use of ML systems for Joint Intelligence, Surveillance, and Reconnaissance (JISR). Currently, there are deficiencies in the time required to analyse large Synthetic Aperture Radar (SAR) scenes in order to gather sufficient intelligence to make tactical decisions. ML systems can assist through Automatic Target Recognition (ATR) using SAR measurements to identify potential targets. However, the advancements in ML systems have resulted in non-transparent models that lack interpretability by the human users of the system and, therefore, disqualifying the use of these algorithms in applications that affect human lives and costly property. Current Deep Machine Learning (DML) implementations applied to ATR are still non-transparent and suffer from over-confident predictions. This study addresses these limitations of DML by investigating the performance of a Bayesian Convolutional Neural Network (BCNN) when applied with the task of ATR using SAR images. In addition, the BCNN is used to perform target detection using data provided by the Council for Scientific and Industrial Research (CSIR). To improve interpretability, a method is proposed that utilises the epistemic uncertainty of the BCNN detector to visualise high- or low-confidence regions in each of the SAR scenes. The results of this research showed that the performance of the BCNN in the task of ATR using SAR images is comparable to current DML methods from literature. The BCNN achieves a classification accuracy of 93.1 % which is marginally lower than the performance of a similar Convolutional Neural Network of 96.8 %. The BCNN outperformed the CNN when the networks were given out-ofdistribution data. The CNN outputs showed over-confident predictions while the BCNN was able to indicate its lack of confidence by using the epistemic uncertainty in combination with the predictive variance in its output. Using the dataset from the CSIR, uncertainty heat maps were generated that illustrated regions of highand low-confidence. The regions with the highest uncertainty were located near large collections of trees and areas near shadows. The high-uncertainty incorrect predictions were fed back into the BCNN, and results showed a reduction in overall uncertainty and detection performance.Dissertation (MEng (Electronic Engineering))--University of Pretoria, 2021.Electrical, Electronic and Computer EngineeringMEng (Electronic Engineering)Unrestricte